Databases have wide spread usage in many large enterprises where high transaction volume, reliability, availability and scalability are of the utmost importance. Typically, these enterprise databases are very large with entries that number into the millions and their reliable operation is critical to the success of the enterprise.
An example of databases with these characteristics is Information Management System (IMS) databases, developed by International Business Machines Corporation. (IMS is a trademark of International Business Machines Corporation in the United States, other countries, or both.)
Due to the high volume of transactions, it is necessary from time to time to re-organize these databases to minimize the performance impact of usage induced fragmentation of the data and related records within the database. Re-organizing a database returns the structure of the database to a non-fragmented state yielding significant performance improvement over an equivalent fragmented database. Those of ordinary skill in the art will appreciate that significant performance improvement for database transactions can be realized by keeping related pieces of information proximate to each other in storage.
However, with large databases comprising millions of records, a reorganization process takes significant amounts of computer resources and, even with dedicated computer resources, may take several hours of elapsed time to complete. These extended time frames may create several problems that the enterprise must deal with. First, the extended reorganization time increases the possibility that a disruptive event or failure could occur prior to completion of the reorganization. It is highly desirable to complete long running jobs without incidence due to the complications of continuing to endure poor performance on the fragmented database, as well as wasted time and effort in restarting (or starting over) the lengthy reorganization process. Accordingly, it is not surprising to find that when reorganization of a large database is required, an enterprise typically assigns very high priority to the reorganization task in order to ensure completion of the reorganization in a predictable and timely manner. This high task priority directs the workload manager (or task scheduler) component of the operating system (or subsystem) to prioritize the availability of critical computing resources for these high priority tasks, thereby facilitating the timely completion of these tasks without unanticipated delays. Examples of critical computer resources typically include CPU cycles, I/O buffers, I/O channels, and the like.
However, simply assigning a very high priority to a task may create another problem. Namely, a very high priority, long running task can tie up critical computer resources for extended periods of time and thereby cause significant disruption to other important work in the system. When online response times undergo substantial degradation, users of the computing system may become frustrated and unproductive. Thus, on one hand the enterprise desires to assign very high priorities for large database reorganizations to ensure timely and predictable completion; but on the other hand, this may be disruptive to users of the database concurrently initiating online transactions with expectations of achieving reasonable response times.
While it is possible to temper the negative effects of these problems by adjusting the priorities of the reorganization task and/or the tasks of other important work in the computer system, this option risks significant delay of the database reorganization which may be unacceptable to an enterprise. That is, if the priority of the database reorganization is lowered relative to other tasks, the resultant effect may be to allow too much computer resource to be diverted away from the reorganization task with the undesirable result of unanticipated, extended delays in realizing the completed reorganization.
Accordingly, there is a need to ensure the timely completion of a critical database reorganization, or other high priority computing process of a very long duration, while avoiding the monopolization of computer resources such that shorter tasks of relatively high priority can complete without extended delays.